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As diverse members of a college of education evaluation committee one of 
our charges is to support faculty as we document and improve our 
teaching. Our committee asked faculty to respond to three qualitative 
questions, documenting ways in which interdepartmental and cross- 
department conversations are used to promote reflective thinking about 
our practice. Three of us investigated the use of CAQDAS to provide an 
additional level of analysis and how we learned more about ourselves as 
scholars through this collaboration. Our findings include 
recommendations regarding the use of CAQDAS to support collaborative 
efforts by diverse scholars. Key Words: Multi-disciplinary Research, 
Multiple Perspectives, CAQDAS, Qualitative Research. 


In this paper we discuss our work on a college-wide evaluation committee, in 
which we investigated faculty’s use of conversation and reflection to demonstrate 
professionalism in teaching. The three of us, as members of the committee, all are 
members of different college departments, and had very different perspectives towards 
research and the use of computers. Linnea is a professor of research methods in a college 
of education, and primarily teaches qualitative research and action research. Linda had 
the most experience with “Computer-Assisted Qualitative Data Analysis Software” or 
CAQDAS, and wanted to bring together Elizabeth and Steve to collaborate on this 
project. Elizabeth is a professor in the department of special education and primarily 
teaches math methods, assessment, and disability studies. Elizabeth has a diagnosis of 
autism and had minimal experience with CAQDAS before this study. Steve is the 
director of assessment for the college and a professor in the technology department. Steve 
has extensive experience with quantitative research and with computers, using computers 
in his quantitative research, but had little experience with qualitative research or the use 
of computers in qualitative research before this study. Significantly, we wanted to make 
visible the ability of CAQDAS to assist diverse scholars in collaborating in research and 
data analysis. We initially conducted the study to highlight the importance of authentic 
assessment of faculty teaching in an era of accountability that often tends to examine only 
quantitative assessments, specifically student course evaluation data. But our further 
work with this data enlightened us to the collaborative benefits of CAQDAS. After 
completing the data collection (described below), completing a traditional group hand- 
analysis of the data, and presenting our findings to our college, Linnea wanted to expand 
the project by inputting the data into CAQDAS. She asked Elizabeth and Steve to join 
her, because of their unique backgrounds. She knew that Steve had extensive computer 
experience, and that Elizabeth had extensive qualitative research experience. But, Linnea 
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wondered how the three of us would be able to work together on a project, and how the 
use of computers might change our understanding of the data. In essence, we wanted to 
expand upon the work of the committee’s group qualitative analysis; but, more 
importantly, we wanted to see if three diverse researchers could work together using 
computer software to assist our collaboration and to see if the three of us would find 
additional insights into one another’s ways of thinking about qualitative data analysis. 

Research Question and Rationale 

We began our inquiry with the following guiding question: 

• When analyzing qualitative data, how will the use of CAQDAS change 
our analysis and interpretations, compared to our first round of coding an 
analysis within our larger committee? 

However, this question changed over the time of our work together. We naively expected 
that using a computer software program to help us analyze might help us uncover hidden 
themes and issues, but Elizabeth was skeptical of this because of her reticence to believe 
that CAQDAS had any value in aiding analysis. Of course data analysis in qualitative 
research can be daunting, due to the sheer volume of languaged, video, audio and 
pictorial data that is available. When we investigated the literature on computer software 
in qualitative data analysis, we changed our question (will follow the rationale) and 
provide the following rationale as a snapshot of that literature search. 

The use of computers to aid in the analysis challenge is fairly new, when 
compared with the history of qualitative research and traditional analysis through hand¬ 
coding, sorting, and memoing. Computer data analysis software for qualitative research 
was available widely to the public in the late 1990s (compared with the traditional 
analysis of qualitative data, which began in the anthropological literature and dates back 
to the work of Boas, 1858-1942; Malinowski, 1884-1942; Mead, 1901-1978 and others. 
It was adopted by other disciplines seeking to document ethnographies of culture in 
education in the 1960s and 1970s with the work of Wolcott, and Geertz, followed by 
Lincoln and Guba, Miles and Huberman, and Corbin and Strauss and others in the 
1980s). Interestingly, although NUD*IST was “born” in 1981 (Richards, 2002), version 
2 in 1987 was still only available on mainframes, and not commercially. It wasn’t until 
1993 that it was available on MACs only, then later on PCs. Atlas ti. released their first 
commercial version of their CAQDAS in 1993 (http://www.qualitative-data- 
analvsis.com/atlas-ti-companv-history.html) and the first English version of MAX QDA 
appeared in 1995 ( http://www.maxqda.com/products/maxqda/historv) . Literature on 
CAQDAS began to appear in the late 1990s, with the majority of literature appearing 
after 2000 (In a search of EBSCO host databases, we found two references for 1996, one 
for 1999, and the rest were post-2000). Most of this literature addressed the capabilities 
of the software and caveats for novice users. Yet we wondered what such programs could 
offer us as three diverse researchers that might support us as we sought to work together 
to present an analysis and interpretation of our study? How might the results be 
expanded? How would the software help us collaborate? What could we learn that will 
facilitate our cohesive interpretations? 
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Theoretical Framework 

In conducting a literature search on the usage and issues of CAQDAS, we 
discovered that with the improvement of software over the last 15 years, authors have 
written increasingly more about the topic in qualitative research journals. Some offered 
caveats or implications for continued use of such software (e.g., Bazeley, 2006; Johnston, 
2006; Lee & Esterhuizen, 2000; Smith & Short, 2001). Lage and Godoy (2008) presented 
a review of literature about the benefits and issues of using CAQDAS. Specifically, the 
authors discuss the literature via a categorization of four themes: (a) “the effectiveness of 
software data management”; (b) “the increase of researcher’s closeness with the date”; (c) 
“the ease of researchers’ communication about the data”; and, (d) “the possibility of bias 
in analytical process” (p. 77, abstract). We were cognizant of these issues as the three of 
us undertook our analysis with computer software following the committee’s previous 
analysis of the data. After reviewing the literature, we confirmed to each other our 
suspicions that we wouldn’t find changes in our analysis (e.g., new themes, differing 
results), as Linnea knew and Elizabeth and Steve soon understood that computer-assisted 
data analysis is largely dependent upon our work in entering and coding the data. We 
wondered, however, after continued searches into the literature, how the software might 
improve our own trustworthiness and confirmability (Lincoln & Guba, 1985) of our 
findings, thereby allowing us to perhaps go deeper into the data, and expand our findings. 
Ryan (2009) wrote about “trustworthiness” in her own study, concluding that while the 
software did not increase rigor for her work, nor change the analysis, it did allow her to 
improve her organizational abilities, and she felt the “trustworthiness” of her work 
increased, due to the linking capabilities of the software in being able to represent her 
analyses and connect various sources of data to interpretations and themes. Ryan 
concluded: “Software programs . . . enable researchers to make visible their 
methodological processes for a more 'trustworthy' study” (p. 158). Ryan’s work 
continued to influence our thinking throughout our analysis, as we felt our diverse 
perspectives made such “trustworthiness” more complex via our different lenses and our 
different approaches to qualitative data understanding and analysis. We expand on this 
issue in our discussion section. 

Davis and Meyer (2009) examined the procedures of analysis, comparing hand 
coding with computer-assisted coding, detailing advantages, disadvantages and common 
assumptions held in the use of computer-assisted qualitative data analysis in sports 
psychology research. Particular caveats included the unpredictable malfunctions of 
computers and the misperception that the program will analyze for the researcher (as 
quantitative software might). They suggested that it is essential that the researcher learn 
the software before using it for a project, and that you cannot separate the researcher from 
the program—that is, the researcher decides which features and codes to use and when; 
the program will simplify organization, recall of data, and visual representation. As 
mentioned earlier, Linnea had extensive previous experience with the software, and Steve 
was computer-proficient in many quantitative analysis programs, so we felt that we had 
sufficient experience to undertake this project. 

Gilbert (2002) further investigated the issue of coding and balancing “closeness” 
and “distance” from the data. We concur that the use of the software left us feeling 
“close” or (for some) overly-familiar with the data, sometimes making analysis difficult. 
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Her emphasis on the “tactile-digital divide” seemed a bit dated for our usage, however. 
None of us had difficulty working with data in a digital manner from a computer usage 
perspective. 

Method of Data Collection/Sampling 

Since our college-level (college of education) committee was charged with 
assisting faculty in the documentation and reporting of the quality of faculty teaching 
(college-level constitutional charge), we chose to survey our faculty (using internet-based 
survey software— http://www.SurveyMonkev.com) about their perceptions of their 
collaborations in reflecting about teaching. We believed that providing faculty with an 
alternative to student-course evaluations, via the documentation of their professional 
activities, would provide faculty with additional materials with which to make their case 
for tenure and promotion. Our tenure and promotion process requires strong 
documentation and persuasive materials to show how we’ve improved our teaching in the 
years leading up to tenure application. We sent out a link to an internet-based survey 
which contained three questions: 

1. What is something you have changed in your teaching recently and why have you 
changed? 

2. Do you have conversations within your program about effective teaching? Please 
provide sample topics. 

3. Do you have conversations with others outside of your program about effective 
teaching? Please provide sample topics. 

We received 39 responses (out of approximately 148 full-time education faculty). 
Data were copied and transcribed from the internet survey software, and initially 
organized by each of the three questions. 

Methods of Data Analysis 

Initially, each of the six members of the original committee (representing six 
different departments in the college of education) was asked to reflect on the responses. 
At our May, 2010 meeting, we chose to break into three groups, each group further 
examining one of the three questions. We came back together as a group to share our 
themes from our group coding, and reviewed all three questions as a group. We agreed 
upon themes from each of the questions, and wrote about it in an executive summary to 
the entire faculty. We chose the themes (Table 1) for each question based upon our small 
group analysis. 
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Table 1. Themes from Hand-Analysis 


Question (n=39) 


Themes from small group hand-analysis 

Question 1: What have you 
changed in your teaching 
and why have you 
changed? 


Technology: faculty are weaving technology into their 
teaching in a wide variety of formats 

Diverse Learners: Faculty are incorporating universal design, 
differentiating instruction, and using literature specific to 
learners 

General Things: Faculty are incorporating theoretical 
frameworks, such as Darling-Hammond; changing 
expectations; chanting texts; using topics outside of texts; 
altering assignments, using current events; incorporating 
more discussion 




Question 2: Have you had 
conversations in your 
department or program 
about effective teaching? 

83% said 

“yes” 

Curriculum: Conversations surrounding curriculum and 
curriculum issues dominated this category 

Pedagogy: Faculty also discussed issues of pedagogy and 
meeting the needs of all learners 

Assessment: Some faculty found issues with assessment in 
their courses—how to accurately and adequately assess 
learning 




Question 3: Have you had 
conversations outside of 
your department or 
program about effective 
teaching? (participants 
indicated they have 
conversations with PK-12 
school personnel, 
friends/family, people 
outside of work, and 
colleagues in national 
organizations) 

80% said 

“yes” 

Our teaching: This topic dominated the findings for this 
question, indicating that many faculty discuss teaching with 
other faculty or others outside of their department or 

program 

The nature of teaching: This indicates that some faculty 
engage in philosophical discussions about what it means to 
teach in higher education and to teach future teachers 

Research, and other tasks related to teaching: Some faculty 
discuss other tasks of being a higher education professor, 
specifically their research and issues related to it 

Policy/issues of education: Some (but few) indicated 
conversations about education policy and other issues in 
education 
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For Elizabeth, this was a new adventure in learning and doing simultaneously, as 
she was learning to use and trust the software, and learning to work with Linnea and 
Steve as collaborators. Not only was Elizabeth skeptical that the software would be 
capable of providing added value, but she had nagging doubts from her traditionalist 
ethnographic upbringing that using technology in any way might “taint” the nature of the 
analysis in ways that “doing it by hand” (tantamount, in her mind, then, to “organically”) 
would not. Elizabeth knew that Linnea was aware of these doubts but also believed that 
talking about them would have the possibility of tainting the data in another direction, so 
for her part, she proceeded with the study in the spirit of seeing what would happen, 
shelving, for the moment, her extremely binary-minded nature. 

We initially broke out the data into the three questions (similar to the original 
committee’s work, as indicated above). However, each of us looked at the responses to 
each question, and began to create “tree nodes” as we noted them in each response. In 
this way computer initial coding is very similar to the work we did as a larger committee 
to analyze the data in that we each assigned meaning to parts of responses. At first the 
meanings, or “tree nodes” were descriptive in nature. For instance, if the respondent said 
that they talked with other colleagues at national conferences about teaching methods 
within a similar course, we would assign that a tree node of “conversations about 
pedagogy”. 

We thought back to Gilbert’s (2002) study. She identified a third stage of coding 
awareness that she calls a “metacognitive shift” (p. 220). During our analysis and coding 
sessions, we found it helpful to include reflection on our own biases—especially in our 
choosing of “tree nodes”. While we felt our “node” choices were descriptive, they still 
represented our “choices”, seen through our lenses. Significantly, ah three of us include 
our lens as “education” faculty. As faculty in a college of education, we are part of 
teacher preparation programs that strive to cultivate reflective thinking in our graduates. 
We would also hope to see the same qualities, such as reflective thinking, modeled by our 
faculty. Gilbert cautions novice software users to reflect “about how and why one works 
in a particular way” (p. 220). It was important for increasing trustworthiness for us to 
acknowledge and reflect upon our biases and the ways that we incorporated our own 
situational awareness as part of the college of education into our analysis of themes and 
issues. 


Discussion 


Interpretations and Findings 

In this section we discuss what we consider to be significant findings for us as 
faculty from diverse departments working on a collaborative project. First, we found that 
a unique aspect of using CAQDAS vs. hand-coding/analysis was the ability of the 
program to create visual representations of data—charts, graphs, trees—that helped us in 
our analysis (we elaborate below) and potentially can help readers to more deeply 
understand our interpretations. Part of this aspect is the ease and agility with which such 
graphics can be made. Second, we discussed how when several of us had coded by hand 
in the past, it was often difficult to justify the work involved in changing and re-arranging 
codings and categories, or to try out new arrangements, due to the nature of “paper and 
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pencil” and the “messiness” that such changes often created, as described below. In 
CAQDAS, it is easy to move things around and try different coding categories and 
groupings. 

A third and final issue we discuss here is related to the nature of the software to 
increase the ability of three diverse researchers—different departments, different teaching 
areas, and different learning styles—to be able to work together to come to a group 
understanding of the issues found within our data. Steve felt that it would be important 
for the three of us to be as diverse as our survey respondents (in his quantitative world of 
reliability and validity), as we were also members of the group (college of education) 
being studied. That we were as diverse in our areas of expertise as our working 
paradigms helped us to represent ourselves and our participants but also caused divergent 
thinking about analysis and themes. However, the use of CAQDAS helped us to 
understand each other’s viewpoints, and work to represent multiple perspectives in our 
analysis. We elaborate on this aspect below. 

In addressing the first issue and thinking about the visual aspects of CAQDAS, 
we offer this example from our analysis sessions. When we were analyzing Question 2 
(“Have you had conversations about teaching within your department or program?”), we 
decided to try a few graphs. We created a tree-cluster by word similarity, and the words 
“curriculum,” “pedagogy,” and “assessment” came up frequently, as we expected based 
upon our initial, group hand-coding. However, the tree node allowed us to visualize 
subcategories by grouping similar words together under the umbrella words. The visual 
offered a striking presentation that illustrated the wealth and depth of the conversations. 
Linnea found the visual helped her to think about all of the themes (nodes) as part of a 
larger collective—almost like a family. Grouped together were phrases from participants 
that are part of teaching or pedagogy, such as: 

• Collaboration with colleagues about curriculum 

• Sharing classroom materials and resources 

• Conversations about program contexts 

• Strategies for teaching 

• Incorporating philosophy and theory into teaching 

• Blending theory into practice 

• Using technology in pedagogy 

• Meeting diverse student learning styles and needs 

These phrases, and more, demonstrated the vast array of topics about which our faculty 
converse, think about, and reflect upon with each other. While we saw these across the 
responses when we hand-coded on several pieces of paper, our intent at that time was 
more of the quality of data ordering (Channaz, as cited in Seidel, 1998, p. 4). However, 
as we were able to return to the mass of data responses in a visual way, via the tree graph, 
we were able to concentrate on the depth of responses around a single theme or issue— 
that of conversations about our teaching or pedagogy. The striking visual display inspired 
our thinking about the reflective opportunity we had given faculty through the tool of this 
survey. 

A second issue in coding is that of “trying out” changes in coding categories. 
While hand-coding, we were hesitant to make many changes, as the volume of data 
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causes a “chaotic moment”, when the data is re-connected randomly, as in pre-coding. 
For Elizabeth the process of creating a graph via the push of a button initially caused a 
“fear” response. She asked, “Wait, can you get the other idea back?” When she realized 
that we could “try out” endless possibilities of visual connections and representations, she 
was excited. We were all impressed with the versatility of the graphics, being able to “try 
out” various graphics, yet still have the group’s foundational analysis from which to 
work. Thinking back to our original work analyzing the data with the use of a group of 
six committee members, we found that most committee members were hesitant to 
question the initial analysis of the pairs of the committee members that we set up. When 
each committee pair brought back their coding categories to the whole group, the group 
accepted them without question, and moved towards a broader picture of the data through 
thematic categorization. However, with CAQDAS we were able to try out different 
themes and key coding categories, without the worry of losing our initial categories. The 
trying out actually is a technique supported by several data analysis theorists, as a means 
of rigorous attention to the data as the foundation (our own thinking and experiences) 
from which we chose our themes (Corbin & Strauss, 2008; Mertens, 2005; Miles & 
Huberman, 1994). CAQDAS allowed us to re-arrange categories, while still keeping our 
initial thematic categories. In fact, we were able to look at them side-by-side in a visual 
representation. We also created visual representations of all of the codes used in a 
question (Figure 1). 

Figure 1. Coding Categories for “What have you changed in your teaching?” 
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Graphics like that in Figure 1 assisted those of us who tend to be more visual, and helped 
us to work at making meaning and understanding within visual learning styles, which 
could then be translated back to language understanding such as is represented within this 
paper. As mentioned earlier, Gilbert (2002) cautioned CAQDAS users to be wary of the 
digital learning curve when using software, and we felt that this particular caveat did not 
apply to us. However, a more applicable caveat that is related to the difference in users as 
mentioned by Gilbert is that of the difference in learning styles—learning that is 
represented by the way we, as researchers, read and understand our data. In writing more 
recently about his seminal Multiple Intelligences (Gardner, 1983) theory, Gardner and 
Moran (2006) posited that “the intelligences can be grouped together for various 
purposes” (p. 228). This is, perhaps, what happened with the use of CAQDAS and the 
visual representations—we, as separate and distinct researchers, with unique learning 
styles, were able to come together and find understanding of each other and the data, as 
well as represent understanding through the software. 

A third finding is that of being able to easily utilize multiple researchers in a 
single project. The ease with which we were able to share suggestions and coding 
categories with each other capitalized on the ability of the software to offer data 
presentation in multiple ways. We discovered that we could incorporate all three 
researchers within one file, share it electronically (via an electronic server), and continue 
coding. However, this was somewhat hampered by the way our university has purchased 
the product—we only have 10 licenses for faculty. The remaining licenses are on the 
server, and can only be accessed in the university computer labs. While one of us had the 
full software on her own work computer, the remaining two researchers had to access the 
software via the computer labs, limiting work time to office time. Since we are a 
university that serves primarily working adult commuters, we have multiple campuses, 
and faculty often teach at multiple locations traveling throughout a metropolitan area, and 
in addition to working from home, these reduced our ability to make use of the computer 
labs. 

Implications 

We realized as we began this project that we probably wouldn’t find “different” 
themes, and this realization was confirmed as we began to work together. We knew that 
the hard work of coding qualitative data still comes from within the mind of the 
researcher and is intricately tied to the perspective of each researcher (Coffey & 
Atkinson, 1996), which in our case proved to be vast, as we each represented vastly 
different learning styles and different perspectives within education. However, we were 
pleasantly surprised at how CAQDAS allowed easy storage and easy access to large 
amounts of language (and other types of) data. We also appreciated the ease with which 
we could manipulate and change around categories, themes and codes. Both of these 
points confirmed Ryan’s (2009) findings of the software’s ability to increase the 
researcher’s organizational abilities. 

Still, using any new software, even for those who commonly work in computer- 
mediated environments, requires time. We found it difficult to negotiate enough time to 
teach the novice members of our group how to work with the software. It was 
enlightening, however, to try different features of CAQDAS and find that these “trial and 
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error” efforts resulted in helping us view the data in new and different ways (i.e., the 
graphs mentioned above) and to understand each other as diverse scholars. The “trying 
out” resulted in unique moments for each of us that expanded our understanding of what 
another member was trying to express related to data analysis or interpretation. Our 
suggestion is for researchers to allow sufficient time to “play around” with the software, 
and to try to look at their data from multiple perspectives. Additionally, we suggest 
diverse scholars can work together through the medium of CAQDAS to provide more 
opportunity for collaboration in data analysis and interpretation. 
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